Transfer Learning for Classifying Spanish and English Text by Clinical Specialties.

Transfer learning has demonstrated its potential in natural language processing tasks, where models have been pre-trained on large corpora and then tuned to specific tasks. We applied pre-trained transfer models to a Spanish biomedical document classification task. The main goal is to analyze the performance of text classification by clinical specialties using state-of-the-art language models for Spanish, and compared them with the results using corresponding models in English and with the most important pre-trained model for the biomedical domain. The outcomes present interesting perspectives on the performance of language models that are pre-trained for a particular domain. In particular, we found that BioBERT achieved better results on Spanish texts translated into English than the general domain model in Spanish and the state-of-the-art multilingual model.

Collecting specialty-related medical terms: Development and evaluation of a resource for Spanish.

BACKGROUND: Controlled vocabularies are fundamental resources for information extraction from clinical texts using natural language processing (NLP). Standard language resources available in the healthcare domain such as the UMLS metathesaurus or SNOMED CT are widely used for this purpose, but with limitations such as lexical ambiguity of clinical terms. However, most of them are unambiguous within text limited to a given clinical specialty. This is one rationale besides others to classify clinical text by the clinical specialty to which they belong. RESULTS: This paper addresses this limitation by proposing and applying a method that automatically extracts Spanish medical terms classified and weighted per sub-domain, using Spanish MEDLINE titles and abstracts as input. The hypothesis is biomedical NLP tasks benefit from collections of domain terms that are specific to clinical subdomains. We use PubMed queries that generate sub-domain specific corpora from Spanish titles and abstracts, from which token n-grams are collected and metrics of relevance, discriminatory power, and broadness per sub-domain are computed. The generated term set, called Spanish core vocabulary about clinical specialties (SCOVACLIS), was made available to the scientific community and used in a text classification problem obtaining improvements of 6 percentage points in the F-measure compared to the baseline using Multilayer Perceptron, thus demonstrating the hypothesis that a specialized term set improves NLP tasks. CONCLUSION: The creation and validation of SCOVACLIS support the hypothesis that specific term sets reduce the level of ambiguity when compared to a specialty-independent and broad-scope vocabulary.

Leveraging PubMed to Create a Specialty-Based Sense Inventory for Spanish Acronym Resolution.

Acronyms frequently occur in clinical text, which makes their identification, disambiguation and resolution an important task in clinical natural language processing. This paper contributes to acronym resolution in Spanish through the creation of a set of sense inventories organized by clinical specialty containing acronyms, their expansions, and corpus-driven features. The new acronym resource is composed of 51 clinical specialties with 3,603 acronyms in total, from which we identified 228 language independent acronyms and 391 language dependent expansions. We further analyzed the sense inventory across specialties and present novel insights of acronym usage in biomedical Spanish texts.

Current approaches to identify sections within clinical narratives from electronic health records: a systematic review.

BACKGROUND: The identification of sections in narrative content of Electronic Health Records (EHR) has demonstrated to improve the performance of clinical extraction tasks; however, there is not yet a shared understanding of the concept and its existing methods. The objective is to report the results of a systematic review concerning approaches aimed at identifying sections in narrative content of EHR, using both automatic or semi-automatic methods. METHODS: This review includes articles from the databases: SCOPUS, Web of Science and PubMed (from January 1994 to September 2018). The selection of studies was done using predefined eligibility criteria and applying the PRISMA recommendations. Search criteria were elaborated by using an iterative and collaborative keyword enrichment. RESULTS: Following the eligibility criteria, 39 studies were selected for analysis. The section identification approaches proposed by these studies vary greatly depending on the kind of narrative, the type of section, and the application. We observed that 57% of them proposed formal methods for identifying sections and 43% adapted a previously created method. Seventy-eight percent were intended for English texts and 41% for discharge summaries. Studies that are able to identify explicit (with headings) and implicit sections correspond to 46%. Regarding the level of granularity, 54% of the studies are able to identify sections, but not subsections. From the technical point of view, the methods can be classified into rule-based methods (59%), machine learning methods (22%) and a combination of both (19%). Hybrid methods showed better results than those relying on pure machine learning approaches, but lower than rule-based methods; however, their scope was more ambitious than the latter ones. Despite all the promising performance results, very few studies reported tests under a formal setup. Almost all the studies relied on custom dictionaries; however, they used them in conjunction with a controlled terminology, most commonly the UMLSⓇ metathesaurus. CONCLUSIONS: Identification of sections in EHR narratives is gaining popularity for improving clinical extraction projects. This study enabled the community working on clinical NLP to gain a formal analysis of this task, including the most successful ways to perform it.

Development and Evaluation of a Serious Game for Teaching ICD-10 Diagnosis Coding to Medical Students.

Objective: Assessment of the pedagogical effect and technological acceptance of the serious game, CODIFICO, which has been designed to train medical students in ICD-10 diagnosis coding. Materials and Methods: We designed the serious game, CODIFICO, as an alternative way to teach ICD-10 diagnosis coding to undergraduate medical students. To assess the pedagogical effect of the game, we used the quasiexperimental pretest-posttest design. The participants began by completing a knowledge pretest on Blackboard. After the pretest, the teacher presented the game to the students and invited them to play it for 1 week. Then, the students completed the posttest on Blackboard. We applied the Wilcoxon test to establish the difference between the pretest and posttest. We designed a questionnaire to evaluate the participants' technology acceptance toward the game. Results: Sixty-one undergraduate medical students from a large Colombian private university took part. There was no statistically significant difference between the pretest and the posttest. However, the game had some positive effects on knowledge. The game was well accepted among the participants. Conclusion: The game, CODIFICO, was useful to teach diagnosis determination, not diagnostic coding. Some of the reasons that caused this situation were insufficient attention to the pedagogical theory, excessive reliance on clinical aspects of the medical training, limited resources, and lack of experience at the medical school to design gamification strategies.

Evaluación automática de la adherencia a las recomendaciones de manejo en falla cardiaca a partir de las historias clínicas electrónicas: herramienta EXEMED / Automatic evaluation of the adherence to recommendations for managing heart failure from the electronic health records: The EXEMED tool

Resumen Objetivo: La implementación de las guías de práctica clínica está limitada por la falta de herramientas que faciliten los procesos de auditoría y retroalimentación a los profesionales de salud. Este estudio evalúa la herramienta automatizada (EXEMED), diseñada para valorar la adherencia a las guías de práctica clínica a partir de la información consignada en las historias clínicas electrónicas. Métodos: En un grupo de 35 pacientes hospitalizados entre enero y marzo de 2016 se evaluó la adherencia a cinco recomendaciones contenidas en las guías de práctica clínica de falla cardiaca del Hospital Universitario San Ignacio. Se utilizó la herramienta automatizada EXEMED y se evaluó la validez de la misma comparando los resultados con la valoración realizada por una junta médica independiente. Se determinó concordancia entre los dos métodos usando el coeficiente kappa. Resultados: La adherencia a las diferentes recomendaciones osciló entre 0% para la determinación del perímetro abdominal al ingreso, hasta 97% para el uso de betabloqueadores al egreso. La proporción de acuerdo entre los dos métodos de evaluación estuvo por encima del 90% para todas las recomendaciones. El kappa para las diferentes recomendaciones fue de 0,78 (IC 95% 0,62-0,95) y 0,64 (0,48-0,80). El tiempo de evaluación se redujo de veinte a dos minutos por paciente con el uso de la herramienta EXEMED. Conclusiones: EXEMED es una herramienta válida y eficiente en la evaluación de la adherencia a las guías de práctica clínica. Se requieren nuevos estudios para evaluar el impacto de su uso asociado a retroalimentación a los clínicos, en la evolución a largo plazo de los pacientes con falla cardiaca.

Abstract Objective: The implementation of clinical practice guidelines is limited due to the lack of tools to carry out audits and provide feedback to the health professionals. In this study, an evaluation is performed using the automated (EXEMED) tool in order to assess the adherence to clinical practice guidelines from the information entered in the electronic health records. Methods: The adherence to 5 recommendations contained in the heart failure clinical practice guidelines was evaluated in a group of 35 patients admitted to the Hospital Universitario San Ignacio between January 2016 and March 2016. The automated EXEMED tool was used to assess this, by comparing the results obtained with the evaluation carried out by an independent medical committee. The kappa coefficient was used to determine the agreement between the two methods. Results: The adherence to the different recommendations varied between 0%, for the determination of the abdominal circumference, up to 97%, for the use of beta-blockers at discharge. Percentage agreement between the two evaluation methods was above 90% for all the recommendations. The kappa for the different recommendations was 0.78 (95% CI; 0.62-0.95) and 0.64 (0.48-0.80). The evaluation time was reduced from 20 minutes to 2 minutes with the use of the EXEMED tool. Conclusions: EXEMED is a valid and effective tool in the evaluation of adherence to clinical practice guidelines. Further studies are required to assess the impact of its used associated with feedback to the clinicians, in the long-term outcomes of patients with heart failure.